Entropy, Combinatorial Dimensions and Random Averages
نویسندگان
چکیده
In this article we introduce a new combinatorial parameter which generalizes the VC dimension and the fat-shattering dimension, and extends beyond the function-class setup. Using this parameter we establish entropy bounds for subsets of the n-dimensional unit cube, and in particular, we present new bounds on the empirical covering numbers and gaussian averages associated with classes of functions in terms of the fat-shattering dimension.
منابع مشابه
A Few Notes on Statistical Learning Theory
2 Glivenko-Cantelli Classes 5 2.1 The classical approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.1.1 The symmetrization procedure . . . . . . . . . . . . . . . . . . . . . 7 2.1.2 Covering numbers and complexity estimates . . . . . . . . . . . . . . 9 2.2 Combinatorial parameters and covering numbers . . . . . . . . . . . . . . . 12 2.2.1 Uniform entropy and the VC dimen...
متن کاملGeometric parameters in Learning Theory
3 Uniform measures of complexity 12 3.1 Metric entropy and the combinatorial dimension . . . . . . . . . 12 3.1.1 Binary valued classes . . . . . . . . . . . . . . . . . . . . . 13 3.1.2 Real valued classes . . . . . . . . . . . . . . . . . . . . . . 15 3.2 Random averages and the combinatorial dimension . . . . . . . . 17 3.3 Phase transitions in GC classes . . . . . . . . . . . . . . . . . . ...
متن کاملEntanglement in quantum spin chains, symmetry classes of random matrices, and conformal field theory.
We compute the entropy of entanglement between the first N spins and the rest of the system in the ground states of a general class of quantum spin chains. We show that under certain conditions the entropy can be expressed in terms of averages over ensembles of random matrices. These averages can be evaluated, allowing us to prove that at critical points the entropy grows like kappalog(2N+kappa...
متن کاملFixed-boundary octagonal random tilings: a combinatorial approach
Some combinatorial properties of fixed boundary rhombus random tilings with octagonal symmetry are studied. A geometrical analysis of their configuration space is given as well as a description in terms of discrete dynamical systems, thus generalizing previous results on the more restricted class of codimension-one tilings. In particular this method gives access to counting formulas, which are ...
متن کاملThe McMillan Theorem for Colored Branching Processes and Dimensions of Random Fractals
For the simplest colored branching process, we prove an analog to the McMillan theorem and calculate the Hausdorff dimensions of random fractals defined in terms of the limit behavior of empirical measures generated by finite genetic lines. In this setting, the role of Shannon’s entropy is played by the Kullback–Leibler divergence, and the Hausdorff dimensions are computed by means of the so-ca...
متن کامل